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Abstract 

In this paper we present a strategy for optimization functions with 
stochastic input. The main idea is to take advantage of decomposition in 
combination with a look-up table. Deciding what input values should be 
used for memoization is determined based on the underlying probability 
distribution of input variables. Special attention is given to difficulties 
caused by combinatorial explosion. 
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1 Introduction 

A basic operation such as calculating a value of a function is in the heart of most 
problem solving processes. In specialized systems (particularly in military and 
other real time systems) where the speed of calculation is of great importance 
and one particular function is a bottleneck, various optimization techniques 
could be applied. There is no general recipe for successful optimization. It 
usually requires problem dependant heuristic, for example: 

• different ways of representing the function 

• hardware implementation or combination of hardware and software im- 
plementation 
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• different algorithm 

• improved data structures used in the algorithm 

• decomposition 

• parallelism 

• pre computed values 

• approximative solutions 

In this paper, we will explore usage of additional resource in optimizing func- 
tions with stochastic input. We will see how to take advantage of functional 
decomposition and present a solution for resource allocation. Several algorithms 
based on dynamic programming are used to deal with the problem of combi- 
natorial explosion. We will also make an attempt to keep the story as general 
as possible. The rest of the paper is organized as follows. Section [5] introduces 
terminology used through the paper. In this section, the problem is presented in 
a formal way and several key questions to be answered are emphasised. Section 
discusses the proposed solution of previously presented problem. Section 0] 
gives additional details of the proposed solution and optimization technique for 
the critical step in the solution. And, finally, section [5] gives conclusion of the 
presented work. 

2 Problem definition 

Let's suppose that we have given a finite commutative ring (U, ©,(g>), where 
U = {m, it2) uk}, K € N. We want to evaluate function / : — > K, 
N G N. We can write /(x) = f(x\,X2, ...,£z\r) where Xi G U for 1 < i < N. 
Computing function value for any specific input vector requires time T c (f). Let's 
assume that we have additional resource of limited size M units. We can use 
one or more resource units to optimize calculation of function value for one or 
more input vectors. 

If we have a memory of limited size M, we would be able to pre compute 
and store function values for up to M values of input parameter combinations 
(input vectors). Assuming that reading a value from the memory requires con- 
stant time Tm and that Tm is significantly less than T c (f), with this approach 
we can cut down the average time of evaluating the function /. Note that the 
term memory in this context can denote a physical memory or a convenient 
data structure. Look-up tables were popular in the world of mathematics even 
before invention of modern computers. Such tables were used mostly to avoid 
manually calculating complex functions (trigonometry functions or logarithms, 
for example) pQ. In computer science, using look-up tables have become stan- 
dard optimization technique in many areas. In designing logical circuits, look-up 
tabls are used because of speed and flexibility, since changing software is much 
easier than changing hardware. In computer programming, memoization is a 
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well-known technique to avoid repeating calculations. In particular, within a 
system, a function could be invoked multiple times with the same input argu- 
ments. Therefore, it would be useful to store computed values, and compute 
from scratch only for those values of input parameters not seen before. Al- 
though this technique is often used by programmers, manually implementing 
such mechanism often requires significant changes in source code and could be 
tedious and time-consuming. That is the reason for some programming envi- 
ronments to provide automated memoization [2J. 

If we don't know anything about input variables Xi , random M combinations 
of inputs (out of K ) could be used for pre-computing. But what if input pa- 
rameters are not of deterministic nature? If the input parameters have stochas- 
tic nature, obviously we can use better strategy for selecting M the most useful 
input combinations. Let's assume that there is an underlying probability dis- 
tribution, so that probability of denoted &S Pij , where Xw=i Pij = 1> 
for each i, 1 < i < N. We will also assume that input parameters have inde- 
pendent distributions. Those distributions could be known in advance, before 
designing the system. Alternatively, distributions of input parameters could be 
learned on-line, during the work of a system which implements the function. 
Information about those distributions could be used to find M most probable 
combinations of input parameters and use them as precomputed and stored val- 
ues. Obviously, that will minimize the expected time of evaluating the function 
which is given by formula: 

Ef[T}= Yl P (x')Tc(f) 

= T M P(X M ) + T c (/)(1 - P{X M )) 
= T c (f)-P(X M )(T c (f)-T M ), 

where 

• Xm is the set of all input vectors used for pre-computation, 

• -P(x') is the probability that an input vector is x', 

• P(Xm) is the probability that an input vector belongs to the set Xm- 

Of course, this is not the only way of using the memory resource. If we store 
function values for M input vectors, we basically did two things. First, we 
reduced average evaluation time. Second, we significantly improved calculation 
for those M vectors. But for all other K — M input vectors, evaluation time is 
still T c . Depending on the usage of the system, this might be satisfying solution. 
But there are some issues in this approach. First, is it possible to use memory 
resource in a different way to reduce average evaluation time even more? And 
second, how can we affect more than M input vectors? One way to approach 
these two problems is functional decomposition. 
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3 Functional decomposition and optimal resource 
distribution 



Definition 1. A decomposition A(/) of a function / is set of functions A(_f) = 
{F,fi,f 2 ...J D }, such that 

/( x ) = f(xi,x 2 , —x N ) 

= F(/ 1 (x 1 ) ) / 2 (x 2 ),...,/ J3 (x D )) 

where components of each vector Xi(l < i < D) are from the set of components 
of the initial vector x. Decomposition traditionally plays important role in 
many areas of mathematics and computer science. It is in the heart of problem 
solving strategy "divide and conquer" and has been particularly significant in 
the areas where parallelism is of great importance. It remains one of the key 
problems in logic synthesis [3] ever since Ashenhurst [I], Curtis [5], Roth and 
Carp [5] did pioneering work in this field, but it has also important applications 
in many other fields of engineering [7J. When combined with look-up tables, 
decomposition is a powerful tool for representation of a function in a more 
economical way. 

Example 1 Let's suppose that we have given function 

h(xi,x 2 ,X3,x 4l ,x 5 ,xe) = xix 2 x 3 + x 4 x 5 x 6 , 

where Xi € {0, 1} . Representing the function h in memory would require storing 
2 6 = 64 values. Now, let's suppose that we decomposed the function h in the 
following way: 

h(xi,x 2 ,x a ,X4 l ,X5,xe) = hi(xi,x 2 ,x 3 ) + h 2 (x4,,x 5 ,x e ), 

where 

hi(x-L,x 2 ,x 3 ) = £1X2X3, h 2 (x 4 ,x 5 ,x 6 ) = x 4 x 5 x 6 . 

Representing functions hi and h 2 would require storing 2 3 + 2 3 = 16 values 
in total. So the function h could be calculated from those 16 values with 
the price of one additional operation +. To optimize a function /(x) by us- 
ing a look-up table (of total size M) in combination with a decomposition 
A(/) = F(/i(xi), /2(x2), /d(xd)), we must find the optimal distribution 
of the memory resource among available functions /.;(xi). The first step in find- 
ing an optimal resource distribution is to calculate average time for evaluating 
the function /. The expected time is given by the following formula: 

D 

E fM ,A[T] = T c (A(f))-J2P(X$\™ j )(Tc(Mx i ))~T M ) (1) 

i=i 

where 

• T C (A(/)) is time of calculating the function / in the decomposition form 
A(/) without using the additional resources, 
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• P(X^j \rrij) is the probability that output of the function fj can be ob- 
tained from memory without calculation under the condition that function 
fj has available rrij memory locations, 

• T c (fj(x.})) is the time of calculating the function fj without using the 
additional resources. 

In order to minimise expected time from equation (TTJ), we are looking to max- 
imise 

D D 

^p(x«K)(m(xj)) - T M ) = 5>jK)< ( 2 ) 

j'=i i=i 

where 

Uj ( mj ) = P(X^\mj)(T c (fj(^)) - T M ). 

So, resource allocation problem can be defined as: find values for m,j,(l < 
j < D) for which the expression Ylf—x Wj(rrij) is maximised while the following 
conditions are satisfied: 

< rrij < M (3) 

D 

Y,™j<M (4) 

i=i 

One way to solve this problem is by brute force - for each combination [mi...m£>] 
which satisfies conditions given by equations (|3|) and ((4]) , calculate the expression 
given by the equation @ and find the best among them. The problem with 
this algorithm is its exponential complexity. By following procedure outlined in 
[5], we obtain the algorithm based on dynamic programming which solves the 
problem in polynomial time. Let's define matrix as 



A[iJ] = max^w fc (m fc ), 



k=l 

where 

j 

w fc = i. 

fe=i 

We will also introduce variable lj as the length (number of components) of vector 
Xj. The following procedure gives the desired solution. 
Initialization. For i — 0, Zi 

A(i,l) = 
B(i,l) = 

Recursion. For j — 2, D and i — 0, min{^^, =1 K lk , M} 

A(i,j + l)= max [A(i',j)+oj j+ i(i-i')] 

i',max{0,i-Jf J+ 1 }<i'<i 
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B(i,j + 1) 



argmax 

i',max{0,i-if i J + 1 }<i'<i 



[A(i',j)+u j+1 (i-i')} 



Stopping and reconstruction. 



argmax 

i,Q<i<min{J2 j k=D K l k,M} 



A[i,D] 



For j = D - 1,D - 2,...,1 



B[i* +1 ,j + l] 



For j = 2,3..,D 



rrij = i* — i 




Time complexity of the described procedure is 0(M 2 D) and space complexity 
is O(MD). 

4 Efficiently calculating Uj(rrij) 

In the previously described algorithm, calculating u>j (rrij ) can be tricky. Brute 
force algorithm gives exponential complexity, so once again the dynamic pro- 
gramming can be helpful. Here, the main challenge lies in the calculation of 
values Pij = P{X^\rrij — i) for each i, < i < L, L — min{M, K lj } and 
finding the corresponding i vectors for which the output of the function should 
be memorized. All those values can be calculated simultaneously by reducing 
the problem to finding L-best paths in trellis. By following procedure from [3], 
we design an algorithm which solves the problem in polynomial time. First, we 
will present the algorithm for finding P{X^j\mj — 1) (i.e. single best path in a 
trellis) and then we will adapt the algorithm to find values P(X^\mj = i) for 
each valid i (i.e. L-best paths). 

4.0.1 Finding the best path in a trellis 

Let's xj = {yi,y2,—,VN y ),yi e Ufori = 1,...,JVjv v . Let * t (i) be the probability 
of the most probable vector of length t (or equivalently the most probable path 
of length t) (t/i, j/2, Vt) for which y t — Uj. For such vector, let's et{i) be 
the value of the component yt— l- The following algorithm gives most probable 
vector and its probability. 

Initialization (t — 1) For 1 < i < K 



*t(») =p(w 
tt(i) = 1 



iM ) 



Recursion (1 < t < N y ) For 1 < i < K 



Mi) 



max [* t _i(i') *p{y t = m)] 

< 1 ' < K 
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e t (i) = argmax[* t _i(j') *p(y t = m)] 

l<j'<K 

Stopping and reconstruction. The probability of the most probable vector is 
given by 

P* = max 

l<j'<K y 

The most probable vector is given by 



(y*N v ) = argmax[*jv y (i')]> 

l<j'<K 



and for t = N y -l,N y - 2...1, 



4.0.2 Finding L-best paths in a trellis 

Let's \&t(i, k) be the probability of the fc-th most probable vector (or equiva- 
lently the fc-th most probable path in trellis) (yi, 1/2, • yt) for which y t — U{. 
For such vector, let's e t {i,k) be the value of component yt-i- The following 
algorithm gives L most probable vectors and their probabilities. Complexity of 
the described procedure is 0(kK 2 N y ). 

Initialization (t = 1 ) For 1 <i < K, 1 < k < L 

=p{yi = Ui) 

e t (i,k) = 1 
Recursion (1 < t < N y ) For 1 < i < K 

tf t (*,fc) = maxW [Vt^ifJ') *p(y t = Ui )\, 

1<3 <K,±<1' <L 

(j*,l*)= argmaxW [* t _i0", * p(Vt = 

l<j'<K,l<l'<L 



e t (i,k) = j* 
r t {i,k) = I* 

where max' k ^ denotes fc-th largest value. 

Stopping and reconstruction. Probability of the fc-th most probable vector 
is given by 

PI = max« [V N (f, I')} 

k 1< 3 '<K,1<1'<L 1 yKJ ' U 

Now, it is easy to obtain fc-th most probable vector: 
(y* Ny ,l* Ny )= argmaxW [* Ny 

l<j'<K,l<l'<L 
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and for t = N y - 1, N y - 2, 1, 

Vt = e (^Wt+i) 
/ t *=r(y* +1 ,Z* +1 ) 

The value P(X^|mj = i) can be calculated by summing the i best proba- 
bilities obtained by the previously described algorithm. 

5 Conclusion 

Stochastic signals appear often in real life. Functions which input has stochastic 
nature are common in real-time systems. In this paper, we have described one 
possible technique for optimization that type of functions. It is based on using 
additional memory resources for speeding up the calculation of functions for 
certain values of input vectors. We proposed dynamic programming procedure 
which can determine the optimal resource distribution for a particular decom- 
position in polynomial time. By using proposed procedure, one could compare 
different decompositions and pick the best one, but discovering various decom- 
positions was outside of the scope of this paper. However, it has been one of 
the hot topics in the science and is certainly an interesting problem for future 
work. 
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